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Office Action Summary 

Application No. 

10/601,350 

Appllcant(s) 

CONNELL ET AL 

Pv2ini 1 nor 

Angela A. Armstrong 

Art Unit 

2654 



~ The MAILING DATE of this communication appears on the cover sheet with the correspondence address 
Period for Reply 


A SHORTENED STATUTORY PERIOD FOR REPLY IS SET TO EXPIRE 3 MONTH(S) FROM 
THE MAILING DATE OF THIS COMMUNICATION. 

- Extensions of time may be available under the provisions of 37 CFR 1 .136(a). In no event, however, may a reply be timely filed 
after SIX (6) MONTHS from the mailing date of this communication. 

- If the period for reply specified above is less than thirty (30) days, a reply within the statutory minimum of thirty (30) days will be considered timely. 

- If NO period for reply is specified above, the maximum statutory period wilt apply and will expire SIX (6) MONTHS from the mailing date of this communication. 

- Failure to reply within the set or extended period for reply will, by statute, cause the application to become ABANDONED (35 U.S.C. § 133). 
Any reply received by the Office later than three months after the mailing date of this communication, even if timely filed, may reduce any 
eamed patent term adjustment. See 37 CFR 1 .704(b). 

Status 

1 )[3 Responsive to communication(s) filed on 23 June 2003 . 
2a)n This action is FINAL. 2b)M This action is non-final. 

3) n Since this application is in condition for allowance except for fomrial matters, prosecution as to the merits is 

closed in accordance with the practice under Ex parte Quayle, 1935 CD. 1 1 , 453 O.G. 213. 

Disposition of Claims 

4) [3 Claim(s) 1-22 is/are pending in the application. 

4a) Of the above claim(s) is/are withdrawn from consideration. 

5) n Claim(s) is/are allowed. 

6) 13 Claim(s) 1-22 is/are rejected. 
?)□ Claim(s) is/are objected to. 

8) 0 Claim(s) are subject to restriction and/or election requirement. 

Application Papers 

9) 0 The specification is objected to by the Examiner. 

10) 0 The drawing(s) filed on Is/are: a)n accepted or b)n objected to by the Examiner. 

Applicant may not request that any objection to the drawing(s) be held In abeyance. See 37 CFR 1 .85(a). 
Replacement drawing sheet(s) including the correction is required if the drawing(s) is objected to. See 37 CFR 1 . 121 (d). 

11) 0 The oath or declaration is objected to by the Examiner. Note the attached Office Action orfonii PTO-152. 

Priority under 35 U.S.C. § 1 1 9 

12) 0 Acknowledgment is made of a claim for foreign priority under 35 U.S.C. § 1 19(a)-(d) or (f). 
a)n All b)n Some * c)^ None of: 

1 .□ Certified copies of the priority documents have been received. 

2. n Certified copies of the priority documents have been received in Application No. . 

3. n Copies of the certified copies of the priority documents have been received in this National Stage 

application from the International Bureau (PCT Rule 17.2(a)). 
* See the attached detailed Office action for a list of the certified copies not received. 


Attachment(s) 

1) H Notice of References Cited (PTO-892) 

2) n Notice of Draftsperson's Patent Drawing Review (PTO-948) 

3) IS Information Disclosure Statement{s) (PTO-1449 or PTO/SB/08) 

Paper No(syMail Date 9/29/03:3/12/04. 


4) 0 Interview Summary (PTO-4 13) 

Paper No(s)/Mai! Date. . 

5) n Notice of Informal Patent Application (PTO-1 52) 

6) □ Other: . 


U.S. Patent and Trademark Office 
PTOL-326 (Rev. 1-04) 


Office Action Summary 


Part of Paper No./Mail Date 05192005 


Application/Control Number: 1 0/60 1 ,350 
Art Unit: 2654 


Page 2 


DETAILED ACTION 
Claim Rejections - 35 USC§102 
The following is a quotation of the appropriate paragraphs of 35 U.S.C. 102 that form the 
basis for the rejections under this section made in this Office action: 
A person shall be entitled to a patent imless - 

(a) the invention was known or used by others in this country, or patented or described in a printed publication in this 
or a foreign country, before the invention thereof by the apphcant for a patent. 

L Claims 1-22 are rejected under 35 U.S.C. 102(a) as being anticipated by Garg et al, 
"Frame-dependent multi-stream reUability indicators for audio-visual speech recognition," 
Proceedings of International Conference on Acoustics, Speech and Signal Processing, ICASSP 
2003, vol. 1, April 2003, pages 24-27. 

2. Regarding claim 1, Garg teaches a method for audio-visual speech recognition 
comprising the steps of selecting between an acoustic-only data model and an acoustic-visual 
data model based on a condition associated with a visual environment (pages 24-26; section 2, 
entitled "The Multi-Stream HDVDVF'; section 3, entitled "Stream Reliability Indicators"; section 4, 
entitled "Reliability Based Stream Exponents."); and decoding at least a portion of an input 
spoken utterance using the selected data model (pages 24-26; section 2, entitled "The Multi- 
Stream HMM"; section 3, entitled "Stream Reliability Indicators"; section 4, entitled "Reliability 
Based Stream Exponents"; Tables 1-2). 

Regarding claim 2, Garg teaches storing the acoustic-only data model and the acoustic- 
visual data model in memory such that model selection is made by shifting one or more pointers 
to one or more memory locations where the selected model is located (Page 26-27, section 5, 
"Database and Experiments"). 
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Regarding claim 3, Garg teaches model selection is based on a likelihood ratio test (pages 
24-26; section 2, entitled "The Multi-Stream HVDVP'; section 3, entitled "Stream Reliability 
Indicators"; section 4, entitled "Reliability Based Stream Exponents"). 

Regarding claim 4, Garg teaches model selection comprises selecting the acoustic-only 
data model when a result of the likelihood test is not greater than a threshold value (pages 24-26; 
section 2, entitled "The Multi-Stream HMM"; section 3, entitled "Stream Reliability Indicators"; 
section 4, entitled "Reliability Based Stream Exponents"). 

Regarding claim 5, Garg teaches the model selection step comprises selecting the 
acoustic-visual data mode when a resuh of the likelihood test is not less than a threshold (pages 
24-26; section 2, entitled "The Multi-Stream HMM"; section 3, entitled "Stream Reliability 
Indicators"; section 4, entitled "Reliability Based Stream Exponents"). 

Regarding claim 6, Garg teaches the threshold value is based on a cost associated with a 
recognition error (Tables 1 and 2; section 3, "Stream Reliability Indicators). 

Regarding claim 7, Garg teaches the hkelihood ratio test is based on one or more 
observations of a given visual feature (Tables 1 and 2; section 3, "Stream Reliability Indicators). 

Regarding claim 8, Garg teaches the given visual featiu^e is associated with the mouth 
region of a speaker of the input utterance (Page 26-27, section 5, "Database and Experiments"). 

Regarding claim 9, Garg teaches the model selection is performed at a rate substantially 
equivalent to an observation rate associated with the audio-visual speech recognition system 
(Page 26-27, section 5, "Database and Experiments"). 

3. Regarding claims 10-22; claims 10-22 are similar in scope and content to method claims 
1-9 and are therefore rejected under similar rationale. 
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Conclusion 


4. The prior art made of record and not relied upon is considered pertinent to applicant's 
disclosure. 

5. Erten (US 2002/01 16197 Al) discloses an audio- visual speech processing system for 
recognizing and enhancing speech by fusing audio and visual speech recognition. 

Any inquiry concerning this communication or earlier communications from the 
examiner should be directed to Angela A. Armstrong whose telephone number is 571-272-7598. 
The examiner can normally be reached on Monday-Thursday 1 1:30-8:00 PM. 

If attempts to reach the examiner by telephone are unsuccessful, the examiner's 
supervisor, Richemond Dorvil can be reached on 571-272-7602. The fax phone number for the 
organization where this application or proceeding is assigned is 703-872-9306. 

Information regarding the status of an application may be obtained from the Patent 
Application Information Retrieval (PAIR) system. Status information for pubhshed appHcations 
may be obtained from either Private PAIR or Public PAIR. Status information for unpublished 
applications is available through Private PAIR only. For more information about the PAIR 
system, see http://pair-direct.uspto.gov. Should you have questions on access to the Private PAIR 
system, contact the Electronic Business Center (EEC) at 866-217-9197 (toll-free). 
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Examiner 
Art Unit 2654 
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May 19, 2005 



